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Abstract 

The objective of this paper is to identify common 
essential information and data needs of colleges and 
universities and to suggest a model to integrate these data 
needs into one office or department. The paper suggests 
there are five major data and information foundations that 
are essential to the effective functioning of an institution: 
(a) accreditation support; (b) analytical reporting; (c) 
assessment; (d) data management and reporting; and (e) 
planning and scanning. Additionally, this paper presents 
the top five tasks within each of the five major foundations 
and offers insight on methods and best practices to 
organize and accomplish these tasks. The paper is not 
designed to provide an in-depth coverage of each data 
foundation. 
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Introduction 

There is extraordinary diversity of thought in 
higher education today in defining and implementing 
organizational structures and methods that best fulfill 
the complex and often overlapping data and information 
needs of a college or university (Volkwein, 1 990). 
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Several factors have simultaneously converged to 
contribute to this paradox including improvements 
in information technologies that have made data 
management and reporting tools more widely 
available throughout an organization (Speier & 
Morris, 2003), increased calls for accountability and 
the resulting need to provide evidence of claims 
(Ruppert, 1 994), and the desire of managers and 
leaders to make data-informed decisions based on 
the "complicated" understanding of multiple rather 
than singular perspectives (Bartunek, Gordon, & 
Weathersby, 1 983). Adding to this complexity is the 
economic crisis, which is notably impacting higher 
education and forcing budget cuts, and consequent 
restructuring, in many institutions (The National 
Center for Public Policy and Higher Education, 

2009). 

The traditional office within higher education 
tasked with organizing data and information 
intelligence is institutional research (IR). The 
definitions of institutional research, and opinions 
on what activities comprise institutional research, 
seem to be as numerous and varied as the over 
4,000 members from 1,500 plus higher education 
institutions who belong to the Association for 
Institutional Research (AIR, n.d.). Saupe (1990) 
provided a succinct and accepted definition of 
institutional research as, "Institutional research is 
research conducted within an institution of higher 
education to provide information which supports 
institutional planning, policy formation and 
decision making" (p. 1 ). Another definition from 
McLaughlin and Howard (2004) states, "The mission 
of institutional research is to enhance institutional 
effectiveness by providing information which 
supports and strengthens operations management, 
decision making, and unit and institutional planning 
processes" (p. 17). Generally, institutional research 
can encompass and provide data and analysis for all 
departments of an institution including enrollment 
services, student life, administrative services, alumni 
relations, and academic affairs. 


While most IR offices can fit under the umbrella 
of the above definitions, Volkwein (1990) noted 
there is tremendous diversity within institutional 
research in structures and tasks, and, therefore, 
there are also inherent ambiguities including 
role contradictions between data demands and 
professional training, whether IR activities are 
centralized or decentralized, where IR belongs in 
the organizational structure, and the role of IR in 
assessment. The office name might include planning 
or assessment or it may be called institutional 
studies. The tasks and responsibilities can range 
from data gathering, analysis, and reporting to 
supplying national survey data, strategic planning 
support, enrollment management support, policy 
support, assessment, accountability, accreditation, 
effectiveness, and information management 
including development and/or maintenance and 
oversight of data warehouses or data marts. A 
comprehensive inventory of IR activities developed 
by Chambers and Gerek (2007) provides a sense of 
the breadth of IR functions. 

The historical diversity of structures and tasks 
within institutional research has also facilitated 
an ambiguity concerning the nature and scope 
of institutional research versus other offices or 
departments. Saupe (1990), in describing the 
expanding and overlapping data functions 
within an institution, pointed out that offices 
of institutional research are often assigned 
responsibilities other than research, such as state 
and federal reporting, providing data for press 
releases, encouraging consistent use of official 
institutional data, providing advice on — and at 
times being centrally involved in — the development 
and operation of computer information systems, 
staying abreast of the literature on higher 
education, and providing advice on planning and 
policy. 

In discussing the evolution of institutional 
research, Volkwein (2008) further states that there 
are strong organizational connections that unite 
what he calls the "golden triangle" of (a) institutional 
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research and analysis; (b) planning and budgeting; 
and (c) assessment, effectiveness, and accreditation. 
Further evidence of the evolving nature of IR was 
provided by Lindquist (1999), who conducted 
a 1 998 survey of AIR members and found the 
topics of high importance to be accountability 
and performance indicators (71%), information 
systems and data management (63%), technology 
issues (63%), persistence and retention (63%), and 
outcomes assessment (61 %). 

While the nature and scope of IR has been 
argued and discussed over the years by those both 
within and outside of IR, other offices outside of IR 
that are concerned with data and information have 
been establishing a foothold within institutions of 
higher education. Today, there are three commonly 
known departments or offices, or some form of 
these offices perhaps under a different name, which 
deal with information and data management and 
reporting functions in colleges and universities 
across America. These are Institutional Research 
(IR), Institutional Effectiveness (IE), and Information 
Management (IM). Assessment may be part of IR 
or IE offices at some institutions, or be a stand- 
alone office at other institutions. Although the 
nomenclature, structures, tasks, and responsibilities 
of these three offices are rarely identical from 
campus to campus, when viewed holistically, most 
data responsibilities can be found in some format 
within one of these three offices. 

The second of the three offices is Institutional 
Effectiveness. Institutional effectiveness has become 
a term of art in higher education, driven primarily 
by the expectations of regional accreditation. It 
hinges on bringing together assessments of all 
components of an institution to provide evidence 
of accomplishing its mission and goals. IE offices 
typically include assessment functions but may 
also include IR functions such as data reporting 
and analysis, designing and administering campus 
surveys, accreditation support, promoting data- 
informed decision making and conducting 
academic program reviews. Some IE offices also 


provide leadership in strategic planning. While some 
institutions limit IE to assessment activities alone, 
regional accrediting bodies generally expect the 
synergistic interaction of activities in assessment, 
other evaluations, data from institutional research, 
strategic planning, and budgeting to provide 
demonstrable institutional effectiveness. For 
example, in allocating an institutional budget to 
various units and activities, or in making budget 
cuts, it is expected that decisions are driven by 
findings of assessment, institutional research, and 
information on progress toward strategic goals, thus 
moving strategic priorities forward and engaging 
in continuous improvement. In this sense, IE 
brings together the other components discussed 
in this paper (i.e., information management and 
institutional research) to assure and demonstrate 
that the institution is meeting its mission effectively. 

The third office is Information Management. The 
scope and definition of information management 
can be as broad and varied as the scope of 
institutional research, and certainly, not all aspects 
of IM apply to the data management discussed 
in this paper. However, a general definition of IM 
responsibility is to develop and manage information 
systems and applications throughout an institution. 
This data management is complex, difficult, and 
inherently linked to IR and IE in two major aspects: 
(a) it is difficult for institutional researchers to 
provide needed reports and analyses if data 
management is flawed, and (b) as the experienced 
users of data, no one is better suited to assist in the 
development of data definitions and structures than 
institutional researchers (Miselis, 1990). Therefore, 
the two components of IM of particular relevance 
to IR and IE are the development and support for an 
enterprise data warehouse, as well as the provision 
of business intelligence and analytical tools that 
make possible timely access to data. These aspects 
of IM are critically related to both IR and IE in 
enabling consistency and centralization of data in 
an integrated manner that facilitates data-informed 
decision making. 
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Objective 

The traditional ambiguity of data and 
information functions within IR has expanded into 
the realm of ambiguity among the offices of IR, IE, 
and IM, with data and information functions at a 
college or university often overlapping between 
the three offices. Therefore, the objective of this 
paper is to identify common essential information 
and data needs of colleges and universities and to 
suggest a model to integrate these data needs into 
one office or department that can synergistically 
create a data-informed institution that eliminates 
duplication, competition, and antagonism between 
disparate offices. The paper suggests there are five 
major data and information foundations essential to 
the effective functioning of an institution that can 
currently be found to varying degrees within offices 
of institutional research, institutional effectiveness, 
or information management. Ensuring that these 
major data foundations are addressed in a coherent 
manner and integrated into one office or under one 
umbrella can create a more effective mechanism 
for fulfilling an institution's information and data 
requirements. 

Framework 

The complexity of data interactions, as well as 
the confluence of organizational structures within 
higher education encompassing data, information, 
and intelligence gathering and reporting has been 
documented and discussed by multiple researchers. 
To mention only a few, Terenzini (1 993) expounded 
the concept of three tiers of organizational 
intelligence: technical and analytical intelligence, 
issues intelligence, and contextual intelligence, 
which acknowledge the complex and ongoing 
changes in tasks and tools confronting institutional 
researchers. Sayers and Ryan (2009) argue for the 
efficacy of creating a "decision support network" 
or an integrated data and information model that 
links the four areas of planning, assessment, data 
gathering, and budget management. Volkwein 


(2008) argues that "professional bureaucracies" (i.e., 
formal, large, structured offices with doctorate- 
level education and significant years of experience), 
which centralize IR activities, are the most effective 
structure for handling institutional intelligence and 
effectiveness. 

In a similar vein, this paper argues that rather 
than decentralize and create separate silos between 
the functions of IR, IM and IE, institutions should 
examine the effectiveness of combining these 
functions under one umbrella. The five major data 
and information foundations of an institution are 
defined in this paper as: (a) accreditation support; 
(b) analytical reporting; (c) assessment; (d) data 
management and reporting; and (e) planning and 
scanning. These five basic foundations are shown in 
Figure 1 and described in more detail in Table 1 . 



Figure 1. Five basic institutional information 
foundations. 


Additionally, this paper presents the top five 
tasks within each of the five major foundations 
and offers insight on methods and best practices 
to organize and accomplish these tasks. The 
paper further argues that if all 25 identified tasks 
are well-implemented and integrated within an 
institution, this will create a truly data-informed and 
data-integ rated institution that can demonstrate 
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Table 1 

Primary Tasks for Five Institutional Information Foundations 


Data Management 
and Reporting 

Analytical 

Reporting 

Planning and 
Scanning 

Assessment 

Accreditation 

Centralize institutional 
data 

Establish a calendar 
of analytical reports 

Develop IR 
strategic plan 

Establish electronic 
system for assessment 
reporting 

Conduct gap 
analysis for regional 
accreditation 

Develop tracking 
system for ad hoc data 
requests 

Develop key 
performance 
indicators 

Be involved in 
development of 
institution strategic 
plan and provide 
environmental 
scanning 

Monitor assessment 
reporting 

Establish an 
evidence repository 

Administer Federal 
and State reporting 
requirements 

Report on historical 
trends 

Develop list of peer 
institutions 

Conduct and analyze 
surveys 

Provide institutional 
and programmatic 
data 

Develop interactive 
online reports 

Utilize research from 
literature and other 

IR offices 

Promote integrated 
campus planning 

Assist departments in 
assessment planning 

Collect and update 
information on 
regional and 
specialized 
accreditation 

Develop a dedicated, 
secure web presence 

Communicate 
findings to key 
decision makers 

Report annually on 
strategic planning 
progress 

Summarize key 
findings from 
assessment 

Establish 
appropriate 
processes for 
accreditation 


institutional effectiveness. The paper provides 
useful, implementable suggestions on creating 
an effective data and information structure within 
an institution. Targeted audiences are those 
establishing a new IR or IE office, those who are 
reorganizing institutional data and information 
organizational structures, and those with existing 
offices who are looking to expand the data services 
they provide. 

Top Five Data Foundations 

Data Management and Reporting 

Data management and reporting are two of 
the most basic, fundamental responsibilities of 
institutional research offices (Wilton, 1990). IR 
offices are relied upon to provide accurate, timely 
data in an understandable format in order to 


support decision making, planning, and policy 
development. In addition to submitting national 
and state survey data, a major role of IR offices is 
responding to ad hoc data requests. Automating 
reports that refresh on a daily or quarterly/semester 
basis can relieve much of the pressure of ad hoc 
requests thereby freeing time for other activities, to 
say nothing of providing consistent numbers. 

However, a classic problem faced by IR offices 
can be described by the old adage "garbage in, 
garbage out." If the data culled from enterprise data 
systems is full of errors or is scarcely populated, the 
resulting report will consequently be full of errors 
and not be reliable. Therefore, in order to build 
trust in IR reports, it is advisable for the IR office 
to have responsibility for data management and 
for ensuring that a data warehouse or data mart is 
established to centralize key management data into 
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one "official" source. This responsibility is typically 
shared with the Information Services/Systems (IS) 
function. The partnership may also include other 
functional areas and/or an information center. It 
is key, however, to have the heavy involvement of 
the IR office with its extensive knowledge of the 
functional aspects of the data. 

Additionally, reports that are produced from 
the IR office should be recognized as the official 
statistics of the institution in order to eliminate 
conflicting numbers. As part of data management, 
both a tracking system to monitor data requests 
and their statuses, as well as a yearly calendar of 
repeating events should be implemented. Lastly, 
in today's digital world, a dedicated, secure website 
accessible from anywhere in the world should be 
established to make reports available 24 hours a 
day. The following are top five activities to actualize 
effective data management and reporting. 

1 . Centralize the collection, management, and 
reporting of institutional data through the 
Office of Institutional Research to ensure 
accuracy and consistency of data. This 
should include the establishment of a data 
warehouse or data marts, as well as a data 
steward group to monitor data policies. 

The concept of a Data Diamond was presented 
at the 2008 AIR Forum (Posey, Few, Elkins, & 

Crosby, 2008). At its core, the Data Diamond 
revolves around the four"c's" of centralization, 
conformity, consistency, and clarity — all monitored 
by a Data Steward Group (DSG) and encompassed 
within a strategic plan. The primary information 
management components that compose the 
foundation of a Data Diamond are (a) the creation 
of clean datasets, including tables, definitions, 
and critical documentation; (b) the building of 
interactive online reports; (c) the development of 
a website with both public and private capabilities 
to create one-stop shopping for data customers; 
and (d) the establishment of a DSG, or an oversight 
monitoring committee for data management. In 


addition to IR and IM, this group should involve 
Student Records, Financial Aid, Finance, Personnel, 
Facilities, and Alumni as key functional areas. It may 
also involve the internal administrative data services 
like Compliance/Risk Management, IS Security, and 
Business Continuity. 

The establishment of a data warehouse or 
data marts as the official source of enterprise data 
can provide the centralization and conformity 
of a Data Diamond. This also goes a long way 
toward providing the rationale and means for the 
elimination of multiple shadow databases, which 
can create both security and consistency issues. 

The data warehouse is basically a repository of 
an institution's stored data and is designed to 
facilitate the retrieval and analysis of data. There 
are a variety of out-of-the box solutions which 
can be implemented, or one may choose to take a 
more customized approach. It is not the purpose of 
this paper to discuss the details and complexities 
of building a data warehouse or which product 
to utilize; rather, the goal is to point out the 
importance and benefit of doing so. There are many 
sources available for research on data warehousing 
including the Higher Education Data Warehouse 
Forum (n.d.) at http://hedw.org . 

Establishing a data management or DSG 
composed of key stakeholder data developers and 
users from throughout the campus will provide 
input, oversight, and monitoring capabilities to 
create and maintain a Data Diamond within the 
institution's data reporting systems. The purpose 
of the DSG is to establish policies and business 
rules for, and monitor efficiencies of, the data 
systems. The DSG should comprise data users from 
throughout the university including administrators, 
faculty, and staff as they all have vested needs for 
accurate data in order to effectively fulfill their job 
responsibilities. As end-users of data, each group 
brings differing perspectives as to what data are 
critical and in what format they should be displayed. 
This broad perspective can be invaluable in creating 
holistic data policies. 
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The DSG can also be responsible for determining 
and assigning access roles to end-users based 
on their need to perform job functions. The DSG 
should be charged with authority to make final data 
decisions. An additional role the DSG members 
can perform is to bring to the forefront knowledge 
of the top reports that need to be developed. The 
frequency of DSG meetings initially may need to be 
as often as once every two weeks but may evolve 
into monthly meetings. 

2. Develop a tracking system for receiving 
and responding to ad hoc data requests. 
Respond in a timely manner and prioritize 
requests. 

The development of an effective tracking 
system should be a priority for all IR offices. Given 
the eclectic nature of requests, the rapid pace with 
which requests can be received, the varying amount 
of time different requests can take to fill, the need 
for follow-up to clarify data requests as well as 
follow-up to ensure the correct data were provided 
and that all questions have been answered, and the 
underlying need to prioritize the importance level 
of each request, a well-designed tracking system is 
critical to the effectiveness of the office. 

The elements of the tracking system should 
be customized for your institution. However, a 
suggested list of variables to track includes the 
following: an issue number, date submitted, start 
date/review date/due date, priority level, assignee 
(who will fill the request), status of the request 
(open, filled, on hold, waiting customer response, 
work in progress), title of request, description 
of request, requestor's name, requestor contact 
information, request type (data output, correction, 
assistance, training, etc.), peer review required/not 
required, time estimate to complete task, budget 
(if applicable), delivery format (Excel, Word, online 
report, etc.), and notes. 

There are a variety of formats for developing a 
tracking system. The use of an Excel spreadsheet 
can be effective for small offices. For larger offices, 


more sophisticated databases or proprietary 
systems might be developed. The key is to find 
a system that works for your office and make it 
a business rule to utilize it for all data requests. 
Having an IR office email for all data requests such 
as datareq@yourinstitution.edu is also useful, 
especially if it can feed into the tracking system. 

You should also consider the development of a 
standard data request form that will allow easy 
synchronization with your tracking system. The 
advantages provided by a proprietary database 
tracking system can make it well worth the time and 
resources needed to provide automation and ease 
of tracking. 

Setting an office business rule for amount of 
time to respond to a request is a good idea. As a first 
step, there should be some acknowledgment sent 
back to the requestor that the request has been 
received. This can be automated or assigned to one 
person in the office as a job responsibility. Tracking 
the start, review, and due dates is important to build 
trust in the IR office. Providing accurate and timely 
data should be the goal. 

3. Administer and file all national, state, and 
local reporting requirements such as IPEDS, 
Petersons, Common Data Set, U.S. News & 
World Report, VSA, etc. This should include 
the development of a yearly activity calendar 
with required dates of submissions. 

The data collection for, and timely submission 
of, national and state reports is a fundamental 
requirement for IR offices. At times, staff may 
view this task as cumbersome or tedious, but its 
importance to the institution should never be 
underestimated. Indeed, in today's digital world, 
the general public, reporters, parents, prospective 
students, and administration and internal staff, all 
can and often do access these reports to gather 
information on an institution. These reports also 
provide a quick comparison between institutions, 
as the reports require standard methodologies 
in collecting and submitting data. Among the 
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more common reports are IPEDS, the Common 
Data Set (CDS), College Board, US. News & World 
Report, Petersons, and the Voluntary System of 
Accountability (VSA) for public universities and 
U-CAN for private universities. Often, the IR office 
simply plays a coordinator role in gathering the data 
and assembling the report, such as the keyholder 
with IPEDS. In this role, IR may fill out the student 
demographic and other common sections, while 
sending other specialized sections of the report such 
as finance and human resources to the respective 
departments to complete and return to the IR office. 

In addition to understanding the important 
function these reports serve, there are three other 
key issues to be aware of with these reports. 

First, timely submission is critical as there are 
usually deadlines associated with submission and 
publication, which means paying close attention to 
due dates ahead of time in order to begin the report 
with enough time to complete by submission. Of 
special note here, is getting the special sections 
to other departments with adequate time for 
them to be able to complete and return to you. 
Second, make sure the required methodologies 
are understood and implemented. The required 
methodology may be different from how you 
report data internally or even on other reports. 

This is important in order to guarantee accurate 
comparisons to other institutions. Third, all data 
should be at least double checked. It is not bad 
policy to require any data reports delivered from 
the IR office to go through at least three pairs 
of eyes before being released, and, in this case, 
because the data are often available to both internal 
and external audiences, it is even more critical to 
thoroughly review the data before submission. 

Lastly, the development of a yearly activity 
calendar should be a standard tool in all IR offices. 
The calendar should include start dates, review 
dates, and due dates for all external data reports, 
as well as internal and external surveys of students, 
faculty, alumni, etc. Also included are analytical 
reports produced on a regular cycle. As a bottom 


line, any activity that occurs on a regular basis that 
has timelines and deadlines should be included on 
the calendar. 

4. Develop interactive online reports that are 

updated daily and by quarter/semester. 

As mentioned earlier, data management 
and reporting are fundamental, indispensable 
responsibilities within a college or university. 
Inherent within this need, the provision of 
accurate and timely data is critical. However, the 
constant, rapid-fire, "I need the data yesterday" 
nature of ad hoc requests can often detract from 
both timely delivery and accuracy. It is therefore 
helpful to design a system that allows for reports 
to be developed in advance without the time 
pressure, which allows for careful thought 
about methodology and for double-checking 
results. It is also beneficial if end-users are able 
to access the reports at any time from any place. 

The development and delivery of online reports 
accomplishes both of these criteria. 

It is not the goal of this paper to discuss which 
tools are best for online report development but 
rather to suggest that to create effective data 
reporting, the best business intelligence (Bl) tool 
for your institution should be researched and 
implemented. The authors' experience is with 
Microsoft Reporting Services and Sequel Server, 
and this tool has proven to be relatively inexpensive 
and effective. Whichever tool is chosen, the reports 
should allow for easy selection by the end-user via 
drop-down menus of parameter selection such as 
year, quarter/semester, program, etc., in order to 
provide an easy method to filter data. 

Security and access should also be given high 
priority in designing an online system. This can 
include encrypting the data that sit on the server, 
as well as utilizing two servers, one for data storage 
and development, and the other for deploying 
the reports to the web. In terms of access, this can 
be assigned by roles or job function (i.e., one has 
access to specific reports as needed to perform 
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job responsibilities). It is also a good idea to have 
all staff go through FERPA training before being 
granted access to reports. 

The prioritization of which reports to develop 
should be based on institutional priorities. 

Be advised that the report development and 
implementation does require dedicated personnel 
with the proper skill set. It is not hyperbole to say 
that these reports will become extremely popular 
and well-utilized throughout the campus. In fact, 
the office will most likely always have a queue of 
reports waiting to be developed. The following are 
some suggested secure reports requiring a staff log- 
in that might be useful on many campuses. 

a. Course enrollments and credit hours. 

These reports can update both daily and 
by quarter/semester and data columns 
can include department/program, course 
number and section, course title, faculty, 
class times, days, credits, room/building, 
room cap, academic cap, number of 
students enrolled, number of seats available, 
number of students denied. Selectors might 
include year/quarter/semester, program/ 
department, and course type. 

b. Application counts, status, demographics. 

A suite of reports can be developed 
for new student applications including 
daily application counts by major, 
application status by major, and applicant 
demographics. The reports might include a 
comparison to the previous two years on a 
day-to-day basis. An "All Application" report 
is also useful for enrollment management. 
This report can drill down to applicant 
name, application received date, status of 
application, student identification number, 
requested major or major, applicant 
demographics, SAT/ACT scores, high school 
information such as GPA, and contact 
information. Parameter selectors can include 
registration status and application class in 
addition to others. 


c. Student headcount, FTE, demographics, 
eligible to register. In contrast to the 
application reports, this set of reports 
deal with continuing, returning, or new 
students. A daily student headcount by 
class, daily headcount by program or 
major, a daily FTE report, and an eligible to 
register report of continuing and returning 
students are examples of these reports. For 
example, the eligible to register report can 
contain student ID number, name, contact 
information, class standing, total credits 
earned, quarter/semester last registered, 
whether there is a registration hold, and 
if tuition exempt. Parameter selectors can 
be continuing student versus returning, 
registration status, graduation status, and 
class level. 

Lastly, an online public fact book and a quick 
facts page should be developed and deployed 
on the public site of the IR website. A quick facts 
page should be a standard on every IR website. The 
quick facts may be only for fall quarter/semester 
and ideally, may even contain a link to the previous 
year's quick facts. Each institution will decide what 
facts to display but a few suggestions are FTE, 
headcount by class, gender, race/ethnicity, full- 
time/part-time status, age breakouts, residency 
status, county of residence, top high schools, 
top community college transfers, faculty counts, 
average class size, total student credit hours, and 
tuition rates. 

5. Develop a dedicated, secure web presence to 
make data and reports available 24/7 from 
anywhere in the world. 

Once data reports are developed, the IR office 
needs a mechanism for displaying reports and 
for making them available 24 hours a day from 
anywhere in the world. This can be accomplished 
very effectively through the creation of a dedicated 
secure IR website. The creation of an IR website can 
provide the centralization, conformity, consistency 
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and clarity of a Data Diamond because the online 
reports will be located in a one-stop website, while 
pulling data from the data warehouse or data marts. 
The online reports should also be designed with 
drop-down menu selectors that allow end-users to 
easily filter and select data. Reports should allow for 
exporting to Excel, CSV format, or pdf and printing. 
Although the online secure data reports and an 
online fact book will be main components of the 
IR website, it also provides a central repository for 
other IR deliverables such as analytical reports, 
survey results, environmental data, quick facts, and 
links to relevant data sources. As end-users become 
comfortable with the site, this will become the first 
place they will look when seeking data. The ultimate 
goal should be to replace static Excel reports sitting 
on a hard drive with interactive updated reports 
accessible on the web. 

Be advised that the creation of an IR website 
requires many skills, resources, and time. The ideal 
scenario is to have the skill and resources contained 
within the IR office because of the constant and 
ongoing need for updates and maintenance. As a 
fallback, the IT or marketing departments usually 
have webmasters on staff and may be willing 
to assist. However, the high-level technical site 
required for institutional research, including the 
need for a secure log-in for staff along with the 
required maintenance, makes it advisable to have 
the web expertise reside within the office. 

Additionally, drafting a project plan that defines 
the parameters of the website is advisable before 
undertaking the website development. What will be 
the navigation and menu system of the site? What 
data will be stored? How and how often will the site 
be updated? These are all necessary components 
of the project plan. Always keep in mind, as a core 
requirement, the usability of the site. Are end- 
users able to easily find the data or information 
they are seeking? A proven method to assist in the 
development of the project plan is to solicit input 
from a wide variety of key data-users. 


Analytical Reporting 

Although the reporting of data statistics through 
spreadsheets and online tools such as Microsoft 
Reporting Services is a vital function of institutional 
research, these reports alone are often not sufficient 
to serve the complete information needs of an 
institution. The provision of value-added, in-depth 
statistical analyses that transform data reports 
into understandable, actionable information is 
often required. These analytical reports should be 
produced on a regular cycle during the academic 
year and be centered on institutional priorities and 
performance indicators or benchmarks. The reports 
should be added to the yearly calendar mentioned 
in the previous section. The indicators and 
benchmarks should be guided by national norms 
and standards available from peer institutions. It 
is important to track historical trends within the 
institution and to summarize changes in these 
trends. And above all, once analyses are produced, 
the results should be effectively communicated 
and explained to key constituencies and senior 
managers and leaders. These analytical reports 
should help to inform assessment activities 
throughout the year. The following are a suggested 
list of the top five activities to implement in order to 
fulfill this important data foundation. 

1 . Establish and implement a work plan of 

analytical reports to be developed during the 
academic year, based on university priorities 
and areas emerging from key performance 
indicators and assessment data. 

The provision of well-designed analytical 
reports to key constituencies should play a major 
role within any IR office. In addition to being of 
high value to the institution, these types of reports 
are the reason why many chose to join the IR 
profession; they are professionally stimulating and 
rewarding. These reports should be added to the IR 
yearly calendar to be produced on a regular basis. 

It is advisable to create templates that facilitate 
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not only the report development on a continual 
cycle but also add an ease of understanding for the 
readers who will become accustomed to receiving 
standardized reports. The templates will provide 
easy production of reports with standard statistical 
techniques, tables, and analysis with updated data. 

Every institution is unique in reporting needs; 
therefore, IR personnel should take the time to 
discover what reports are most needed within their 
specific institution. The following are a "Top 5" list of 
analytical reports provided only as a guideline but 
are certainly reports found to be of value at many 
institutions. 

• Retention/Persistence/Graduation studies 
with odds; 

• Space utilization and management reports; 

• Productivity reports (faculty workload, grants 
and research, student credit hours); 

• Enrollment management reports with 
odds of enrollment and characteristics of 
applications and enrollment; and 

• Finance/ revenue reports (revenues and 
expenditure trends). 

2. Develop key performance indicators to 

measure effectiveness compared to peers 

and national standards. 

Setting key performance indicators (KPIs) 
is necessary in order to measure institutional 
effectiveness. The measurement of the KPIs aids 
both accountability issues in accreditation as 
well as informs institutional improvement in 
assessment. However, just as the identification 
of peer institutions can be a political process and 
therefore presents difficulty in reaching consensus, 
obtaining agreement around the identification of 
key performance indicators can also be an intense 
political process. In particular, close attention 
needs to be paid to the methodologies selected to 
measure the indicators. For example, if it is agreed 
to have the student/faculty ratio as one of the 
performance indicators, administration and faculty 
might have different ideas on how and who to 


count as faculty. This is where it might be advisable 
to adopt a standard methodology from one of the 
national reports such as the Common Data Set. 
Additionally, the identification of where the data 
will come from to measure each KPI should be 
defined at this development stage. 

The KPIs should be established for the institution 
but certainly can be a part of unit assessment plans 
as well. For the institution-level KPIs, it is advisable 
to involve a broad, representative spectrum of 
constituents from throughout the college or 
university community. This broad involvement 
ensures that the KPIs will be the most relevant 
and important measures to track. The KPIs should 
be tied to existing plan goals in order to ensure 
relevancy. The State Higher Education Executive 
Officers (SHEEO) website provides an excellent 
reference site for state accountability measures and 
performance indicators (http://www.sheeo.org/ 
links/links_results.asp?regionlD=53&issuelD=2). 

A mechanism or mechanisms need to be 
defined as to how and when progress on the 
KPIs will be reported. This could be as seldom as 
within an annual report or as often as a continually 
monitored dashboard available on the IR website. 
Somewhere in the middle may be quarterly/ 
semester reports that advise budget and other 
policy decisions. 

A helpful tool to create a balanced set of 
indicators that recognizes the many dimensions 
that contribute to a successful organization is one 
borrowed from the world of business, aptly named 
a balanced scorecard. This tool was introduced 
by Robert Kaplan and David Norton in 1 996 in 
th e Harvard Business Review in recognition of the 
fact that focusing solely on the financial aspect 
of an organization is insufficient for success. The 
balanced scorecard identifies four perspectives that 
an organization should consider in planning and 
assessing performance: (a) financial, (b) customer, 

(d) business processes, and (e) learning and growth. 
This tool has been used successfully in institutions 
of higher education, either in the original format 


Page 12 


AIR Professional File, Number 126, Integrating the Functions of Institutional Research 


/HR 


or as a modified version, such as that described by 
Carpenter-Hubin and Hornsby (2005). 

3. Monitor and report on historical trends 
within the institution and summarize key 
findings that merit high-level consideration 
at the institution. 

With the constant demand upon IR offices to 
produce current data reports, the value of historical 
trends can sometimes be overlooked. However, it 
is within historical trends that the most informative 
data can often be found. Only by looking at data 
in the context of trends does one gain a sense 
of importance or relevance. What may initially 
appear to be alarming or encouraging may well be 
tempered by a 5- or 1 0-year perspective. The trend 
data reports also provide an easy-to-find source of 
historical data. 

Because it generally does not require much 
more effort to generate several years of data than a 
single year, one should always consider gathering 
several years of data, if available. For example, if 
the data are extracted using SQL coding, why not 
simply write the code for several years instead of 
one? This is a good "best practices" business rule to 
establish for data coders. 

The historical trends are also great candidates 
for charts, which can graphically spice up any report 
or website in addition to providing the rich and 
informative data. Most software programs today 
have relatively easy-to-use graphing capabilities. 
These trend charts also are often required in self- 
studies and accreditation reports. 

4. Utilize research already available by keeping 
up with IR literature and seek assistance 

of experienced members of IR community 
nationally and internationally to inform your 
work on research studies. 

There is a wealth of guidance available for 
developing quality analytical reports. Although 
each IR office certainly needs to determine the 
methods, techniques, and reports that are most 


suitable for its respective institution, there is no 
need to reinvent the wheel. By actively reading 
published literature, one can often find existing 
studies and models to apply to your reports. A great 
source is published literature available through 
online databases usually available through an 
institution's library. Another recommended source is 
Pascarella and Terenzini's updated 2005 work, How 
College Affects Students: A Third Decade of Research 
(Vol. 2). 

Joining professional organizations can also 
provide a tremendous source of guidance and 
assistance. The Association for Institutional Research 
(AIR), The Society for College and University 
Planning (SCUP), and The Higher Education Data 
Warehousing Forum (HEDW) are only three of the 
various professional organizations from which 
one can benefit through networking and shared 
expertise. Additionally, there are many electronic 
mailing lists to which one can subscribe to in 
order to benefit from the expertise of colleagues. 
The SPSS [SPSSX-L@LISTSERV.UGA.EDU] and 
SAS [SAS-L@LISTSERV.UGA.EDU] listserves, the 
HEDW [hedw@Princeton.EDU] listserve for data 
warehousing, and the Reshaping IR [RESHAPING_ 
IR@LISTSERVCSUFRESNO.EDU] listserve for IR 
practitioners are only a few examples. 

Lastly, assuming an office has established 
a budget for training, there are a variety of 
associations and companies that provide training 
on a contract basis. In addition to increasing the 
skills within an IR office, this training can be a 
positive motivator for staff who benefit from the 
professional development. All of the large software 
companies provide some version of training, 
either on location or online. There are also many 
consulting companies who conduct specialized 
training as well. There are an ever-increasing 
number of online webinars also available, some 
free and others at minimal cost. This includes those 
provided by AIR http://www.airweb.org/?page=685. 
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5. Effectively communicate findings of 

analytical reports on actionable items to key 

decision-makers and campus community, as 

appropriate. 

As basic as it sounds, a key action needed to 
ensure the success of an IR office is the effective 
communication of analyses to key stakeholders. If 
produced reports simply sit buried in the IR office or 
even on a web page, all the effort and hard work of 
the office will be for naught. Establishing standard 
procedures to deliver analytical reports should be a 
business rule within every IR office. There are several 
methods available to most offices to effectively 
employ this strategy. First, every institution usually 
has email lists of existing committees. Make a list 
of relevant committees and groups on campus 
and email reports to those committees to whom 
the work is relevant. There is no need to be overly 
discriminating here; often an analytical report 
can be inconspicuously useful to a committee 
that might not seem to have a direct interest. And 
promotion of the IR office's work is a good thing. 

The senior management team should always be 
included among these groups. 

Second, be proactive in asking for time on 
committee meeting agendas. The conventional "dog 
and pony" show can be quite effective when sharing 
results. In addition to creating valuable face-to-face 
interactions, live presentations of data can provide 
the opportunity for the audience to ask questions, 
thereby clarifying any possible confusion around 
the report. 

Lastly, always post your reports on your 
dedicated website using a clearly defined menu 
that simplifies an end-user's ability to locate the 
report. It is advisable to have a highly visible space 
on your website dedicated to "What's New" or 
"Newly Published Reports." It can also be effective 
to convene occasional meetings of known end- 
users of your data, to walk them through your 
website and demonstrate how to find data within 
the site. 


As a bottom line, be proactive in getting your 
analytical reports out in as many ways as possible to 
as broad an audience as possible. 

Planning and Scanning 

Strategic planning and the accompanying 
environmental and internal scanning required 
for effective strategic planning are extremely 
important functions for an IR office to perform. 

In addition to providing underlying institutional 
data from assessments, scans, analytical reports, 
peer and national benchmark comparisons, and 
data from enterprise data systems, the IR office 
can bring a holistic institutional understanding to 
inform the strategic planning process. In fact, it is 
the strategic planning process that ties together 
all the activities in which the IR office is involved. 

The IR office can also provide a critical function in 
helping to integrate all planning across campus. 

This can include identifying cross impacts and 
tracking linkages to budget. It is also important for 
the IR office to stay involved in strategic planning 
by helping to monitor progress on the goals of 
the strategic plan. This not only provides valuable 
assessments but also maintains the strategic plan 
as a living document, rather than something that 
simply sits on the shelf. In addition to supporting 
the institutional strategic planning process, the IR 
office needs to develop its own strategic plan that 
clearly defines its roles and responsibilities in order 
to effectively fulfill its function within the institution. 
McNamara (n.d.) provides a good overview of 
strategic planning including understanding 
strategic planning, implementation, benefits, and 
basic models. 

1 . Develop an IR office strategic plan that 

clearly articulates mission, values, goals, and 

objectives in order to assess the progress of 

the IR office. 

Although support for strategic planning at the 
institution level is a common role for IR offices to 
perform, how many IR offices develop their own 
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strategic plan?The importance of this cannot be 
overemphasized. A significant barrier to the success 
of an IR office can occur when the mission and goals 
of the office are not clearly articulated because 
this can enable "mission creep" (i.e., stretching an 
office beyond its resource capabilities through 
requests for projects and services outside of the 
defined scope of the office). As discussed earlier in 
the paper, the definitions of IR are wide and broad, 
and a nebulous definition of IR responsibilities 
can lead to constituents placing unreasonable 
demands upon the office. The solution to this is the 
publication and promotion of an IR strategic plan 
that informs everyone of the scope and mission of 
the IR office. 

The strategic plan should be developed in 
concert with senior administration and other key 
data users and should be based upon the mission of 
the institution to ensure that the IR office is relevant 
to the effectiveness of the institution. When there 
is strong linkage between the IR mission and goals 
and the mission and goals of the institution, all 
reports and services provided have a better chance 
of being utilized. In other words, it is up to the IR 
office to clearly define the meaning of IR at your 
institution. 

There are many existing models of strategic 
plans to review and adapt. One suggested source 
of reading is the Higher Education Resource 
Hub (n.d.) website (http://www.higher-ed.org/ 
strategic_plans.htm), which provides links to various 
strategic plans as well as other strategic planning 
resources. After studying various plans from other 
institutions, a plan customized for the IR function 
at your institution can be crafted. At a minimum, 
it is suggested that the plan include a mission 
statement, values, vision, a SWOT (strengths, 
weaknesses, opportunities and threats) analysis, 
and goals/objectives/metrics. The goals section 
should also contain the calendar of critical IR 
timelines for reporting and surveys. 

Assessment of the IR strategic plan should be 
conducted on an ongoing basis. If IR is the office 


preaching assessment, then it is purposeful for 
IR to lead the way by assessing itself. Customer 
satisfaction can be measured through surveys, as 
well as focus groups. Additional assessment can 
be conducted through analysis of data delivery, 
timeliness of delivery, and implementation of stated 
goals. 

2. Be involved in the development of the 
institution's strategic plan and provide 
relevant institutional data to support the 
development of the plan including a SWOT 
analysis. Conduct environmental scans and 
perform surveys of stakeholders to support 
strategic planning efforts including new 
program development. 

The development of the institution's strategic 
plan is an important activity that should be the 
guiding influence to enhance, and perhaps, 
transform the institution. It is an activity in which 
IR should be involved because the use of relevant 
data, the knowledge of environmental factors 
that may impact the institution, and the broad 
participation of stakeholders in charting the course 
for the institution's future are all hallmarks of a 
useful strategic plan, and these are areas in which 
IR has expertise. There are some foundational 
institutional data that should be considered in 
developing a strategic plan and could serve as 
benchmarks. They include the following: 

• Student trend data on enrollment, 
degree production, retention, and 
graduation; 

• Characteristics of students entering the 
institution (demographic and academic 
performance); 

• Data on academic program productivity/ 
instructional activity; 

• Fiscal data including allocation/ 
expenditures by divisions and programs; 
endowment information; 

• Faculty numbers, characteristics, and 
distribution among academic areas; 
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• Research and scholarly activity (if 
appropriate to institutional mission); 

• Space and physical facilities information 
to assess current status and gaps; and 

• Issues revealed through assessment, 
including institution-wide surveys 
such as those relating to engagement, 
opinions on institutional services, and 
basic skills. 

In addition to institutional data, it is advisable 
to conduct a SWOT analysis to determine strengths 
that the institution may capitalize on and further 
develop, weaknesses that must be addressed in 
order for the institution to advance, opportunities 
in the environment that could be utilized, as well 
as plan for mitigating potential threats to the 
institution. A related activity is conducting an 
environmental scan, which can include an analysis 
of competitors, market research, and analysis of 
viability of contemplated new programs. To have 
wide participation and buy-in for the plan, it is 
essential to seek input from stakeholders on the 
SWOT analysis and allow them to provide their 
insights and perceptions related to the direction of 
the institution. IR can also be effective in helping 
to state goals in measurable terms so that the 
implementation can be managed and assessed 
effectively. Measurable goals should also have 
benchmark data against which to assess progress. 
Some institutions find SMART (specific, measurable, 
attainable, realistic, timely) goals useful in strategic 
planning. The utilization of a balanced scorecard as 
mentioned in the previous section is also useful for 
assessing progress within a strategic plan. 

3. Develop a list of peer institutions. 

As mentioned previously in the paper, 
the development of inter-institutional peer 
comparisons plays a large role in establishing 
benchmarks and assessing progress in meeting 
these benchmarks. The availability of comparative 
data will add validity to policy development and 
operational decisions that affect salary and budget 


decisions. There are four major groups of peer 
institutions that can be identified: peer, aspirational, 
competitor, and predetermined (Teeter & Brinkman, 
2003). Perhaps the most common groupings and 
those best utilized for comparative data reports are 
the peer group that identifies similar institutions 
and the pre-determined group based on athletic 
conferences, history, or geography; although, there 
is certainly high value in identifying aspirational and 
competitor peers. 

Be forewarned that the establishment of peer 
groups can be one of the most political issues an 
institutional researcher will encounter. It is not 
uncommon for the IR office, top administration, 
deans and faculty, and governing boards to all 
have different ideas of who an institution's peers 
should be. Therefore, it is advisable to establish 
a well-thought-out approach to researching and 
establishing the peer comparison institutions. 
Although any selection method will be open to 
political attacks and agendas, the following is one 
suggested method that can help minimize attacks 
and gain consensus. 

As a first step, create a list of possible variables 
to use in comparisons. The list should include all 
known factors that influence campus culture and 
decision making. The list might include variables 
such as the total number of students, ratio of 
underrepresented minority students, tuition cost, 
program mix, faculty salaries, amount of grants and 
research, geographic location of campus, etc. With 
some inquiry and research, it is relatively easy to 
generate a list of 40 to 50 variables. 

Once the list is completed, it can be placed into 
a spreadsheet format and presented to a broad 
group of key constituencies to rank or weight the 
variables in order of importance. For example, many 
institutions have some type of leadership council of 
campus leaders including deans, directors, students, 
and top administration. The variable spreadsheet 
can be presented to members of this council to 
rank the top 1 0 variables in order of importance, 
both for peer and aspirational peer comparisons. 
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The results can be tabulated to create a weighting 
system. The tabulated results should then be re- 
presented to the leadership council for confirmation 
and tweaking. With agreement on the variables, the 
IR office can then go about finding peer institutions 
based on the agreed-upon variables of importance. 
The resulting peer list can then be presented to the 
leadership council for discussion and affirmation. 

At the September 201 0 Southern Association 
for Institutional Research (SAIR) conference, 
McLaughlin, McLaughlin, and Howard provided 
a detailed explanation of a process to select 
variables and apply weights in order to create peer 
institutions based on nearest neighbors. 

4. Promote the development of aligned campus 
planning including links between unit plans 
and the strategic plan, as well as integrating 
all major campus plans to show links and 
impacts between campus initiatives (finance 
plan, academic plan, student life plan, 
development plan, and strategic plan). 

Although the concept of integrated planning 
tied to budget has existed for a number of years 
in the business environment, there is not much 
evidence of its effective implementation in higher 
education. Integrated planning, in general, refers to 
the connection of planning functions that include 
technologies, applications, and processes across 
an institution to improve institutional alignment, 
student success, clarity of direction, and financial 
performance. Most institutions have a variety of 
major plans on campus ranging from an academic 
plan, a student life plan, a facilities capital plan, 
an advancement plan, an enrollment growth 
plan, a finance plan, and a strategic plan. While 
it is commendable that individual departments 
develop plans detailing major initiatives, if these 
plans are developed in isolation or do not take into 
consideration the impact on other plan initiatives 
and the total impact on budget, an institution 
may find itself in an antagonistic position with 
conflicts between departments. Additionally, as 


the current economic crisis and resulting budget 
cuts have shown, if institutional priorities are not 
clear, making budget cut decisions can be quite 
difficult. Traditionally, strategic planning was 
supposed to facilitate this planning, and while 
the concept of strategic planning has merit, the 
successful application of the plan is often lacking. In 
reality, strategic planning often does not drive the 
critical decision-making process. Leaders can make 
decisions using incomplete information, which can 
cause sub-optimal resource allocation. While each 
functional department silo is often disconnected at 
the strategic level, the disparate plans are indeed 
connected at the operational level. 

There is a need to create a comprehensive 
understanding of strategic directions, in other 
words, to create a clear path from decision making 
to institutional priorities related to mission and 
budget. This requires a clear understanding of an 
integrated relationship of major campus plans 
and supporting plans. Because this requires cross- 
institutional understanding, this is a key function 
that the IR office can perform. Although this is not 
a traditional role of IR, who else is better situated to 
serve this critical function? The IR office can not only 
record the overlaps and impacts between campus 
plans but can also track and monitor impacts to 
budget. 

An initial step to promote integrated planning 
can be to call all the major plan stewards and senior 
administration together for a one-day retreat to 
discuss the implications of non-integrated planning. 
Have each plan steward record his or her respective 
plan initiatives year by year over the next five or six 
years. Then put all the initiatives together by year. It 
becomes graphically clear that each plan initiative 
may impact other plan initiatives. For example, if the 
academic plan calls for the introduction of a new 
high-demand major the next year, the following 
questions may arise: How will this impact the student 
life plan? What additional facilities will be required? 
What role might the advancement office play? 
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5. indicators and goals and provide an annual 

report on this progress to institutional 

leaders. 

A strategic plan is only as useful as the actions 
it generates. Equally as important as developing a 
strategic plan is the development of a systematic 
mechanism to monitor progress on the goals of the 
strategic plan. This task may fall within the purview 
of the IR office. For successful implementation, each 
goal within the strategic plan should have these 
elements: 

• a responsible party identified for its 
implementation, 

• a timeframe, 

• anticipated results, and 

• assessment of the actual results. 

As discussed above (item #3 under Planning), 
the units should have unit-level strategic plans or at 
least goals that link to the institutional strategic plan 
goals for which they are responsible, and should 
report on results of their actions on the goals on an 
annual basis. IR may decide to use a commercial 
product to gather and monitor such planning 
information, or develop one in-house. Either way, 
it is essential to monitor progress on the strategic 
plan in order to have a truly effective strategic plan 
that moves the institution forward. IR would need 
to assess this progress based on the information 
gathered from the various units responsible for 
the implementation of each goal. The institutional 
leaders need to be apprised of the progress so 
that determinations can be made for changes in 
strategies, in allocation of resources, or, sometimes, 
in modification of goals that are informed by the 
feedback loop during implementation. Strategies 
from the world of business, such as the balanced 
score card, can be adapted to determine the success 
and need for modifications along the way. 

Assessment 

Assessment has become a cornerstone of 
both regional and specialized accreditation within 


the past two decades. Regional accrediting 
bodies expect that the institution regularly 
assesses whether it is meeting its outcomes in 
all aspects of its operation, including financial 
solvency, academic programs, student support, 
educational support, and administrative units. 

This includes assessment of central activities that 
cut across units or even the entire campus, such 
as general education. Specialized accrediting 
bodies also expect systematic assessment of 
various components of the programs undergoing 
assessment. It is imperative that an institution have 
a formal, institution-wide assessment system that 
is documented consistently, over a period of time. 
This is not an activity that can be undertaken simply 
during the year of accreditation. Accrediting bodies 
expect that assessment activities are a normal part 
of how the university operates, are continuous, and 
result in demonstrable improvements over time. A 
central office, such as IR, needs to have the authority 
to institute, monitor, and make revisions to an 
assessment system. The essential activities that 
should be undertaken by a central unit on behalf of 
the institution include the five activities that follow. 

1 . Establish an electronic system for regular 

reporting of assessment plans and results. 

Include elements required for your institution's 
regional accreditation and sufficient flexibility to 
adapt to specialized accreditation assessment 
requirements as well. Critical elements that all 
accrediting bodies require for assessment include 
identification of desired outcomes, measures of the 
outcomes, analysis of the measurement results, and 
improvements based on the results. The electronic 
system should capture, in summary form, the 
elements specified by the regional accreditation 
body for the institution. While the system may be 
built to meet the regional accreditation requirements 
for reporting, it should have sufficient flexibility 
that programs with specialized accreditation may 
meet their accreditation requirements with respect 
to assessment as well, with one activity, rather than 
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two sets of assessment activities to satisfy regional 
and specialized accreditation. Several commercial 
products are now available to document assessment 
activities. Alternatively, an institution may develop 
its own system. 

2. Monitor reporting into the system to ensure 

universal participation. 

It is important to establish a university 
mechanism for regular review, feedback, and 
evaluation to ensure assessment is meaningful 
and resulting in improvements. In an institution's 
early years of assessment activities, considerable 
monitoring is required to ensure that all units are 
participating. Faculty who are accustomed to 
perceiving themselves as completely independent 
agents are unaccustomed to collective discussions 
about assessment results that demonstrate the 
achievements of students in aggregate. Faculty 
are now required to work together to identify 
program outcomes and to demonstrate that 
students are indeed achieving those outcomes. For 
programs that undergo specialized accreditation, 
the assessment should address the outcomes 
and elements specified by the accrediting body, 
if any. The monitoring and review should ensure 
that all of the steps of the assessment system are 
being completed and followed, including making 
improvements. In the absence of this last essential 
step of "closing the loop" by making improvements 
based on assessment results, one has not truly 
engaged in systematic assessment and would not 
be able to demonstrate institutional effectiveness. 

The monitoring, review, and feedback could be 
done by IR staff, if sufficiently knowledgeable, or by 
a committee of appropriate individuals including 
faculty, convened by IR. It is advisable to have 
an assessment committee on campus so that the 
ownership and expertise is not simply concentrated 
in IR. Committees may also be identified to be 
responsible for assessment activities that cut across 
units, such as general education or a Foundations 
of Excellence (http://www.fyfoundations.org) 


review of first-year programs. In such activities, 
the committees would document that students 
are meeting the learning outcomes identified by 
the institution for general education (sometimes 
referred to as the core) or Foundations of Excellence 
through assessment activities and recommend 
improvements based on analysis of assessment 
results. While the committees may not assess 
students directly, they would analyze student 
products (perhaps from courses involved in general 
education or the Foundations of Excellence) and 
develop an assessment report for the activity. 

3. Conduct national and in-house surveys at the 
institutional level for assessment purposes, 
such as the National Survey of Student 
Engagement (NSSE), student exit surveys, 
alumni surveys, feeder institution surveys, 
external community surveys, and faculty/ 
staff/student satisfaction surveys. 

Assessment requires both direct and indirect 
measures. The indirect measures include surveys 
and usually include those listed above. Participating 
in NSSE/FSSE and other national surveys provides 
a convenient means of comparing performance of 
the institution to that of peer institutions. They may 
be administered about once in three years in order 
to track changes (more frequent administration may 
not provide enough new information to outweigh 
the cost of administering the survey). Some surveys 
may be developed in-house. It is advisable to 
include the campus assessment committee, or a 
campus survey committee, in these decisions and 
even in the development of local surveys. Beware 
of assessment and survey fatigue, which will make 
it difficult to convince respondents to complete 
yet another survey. It is therefore necessary to be 
strategic in which surveys are to be utilized and 
obtain the most important information from them. 
IR will probably be responsible for administrating 
the institution-wide surveys, and IR may be called 
upon to assist units to devise and analyze their own 
surveys for the unit assessment reports. 
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Examples of national surveys widely used for 
assessment are available on the National Center for 
Postsecondary Improvement website (http://www. 
stanford.edu/group/ncpi/unspecified/assessment_ 
states/instruments.html#institutionalEffectiveness). 
[Note to compositor: In the bulleted list below, please 
remove the embedded URLs. They are old links that 
return error messages, and the information is provided 
in the URL above.] 

The instruments listed on the site are organized into 
four categories: 

• Institutional Effectiveness (surveys the 
opinions and institutional experiences of 
the students, faculty, staff, administrators, or 
alumni); 

• Basic Skills (measures the general cognitive 
capacity of the students); 

• Affective Development (surveys the values 
and social development of the students); and 

• Major Field Exams, including Biology, 

Business, Chemistry, Computer Science, 
Criminal Justice, Economics, Education, 
History, Literature in English, Mathematics, 
Music, Physics, Political Science, Psychology, 
and Sociology. 

AIR also has several assessment resources listed 
on its website (http://www.airweb.org/Resources/ 
Links/Pages/LinksQualityAssessment.aspx). 

4. Assist all campus departments in the 
development of assessment plans 
including outcomes and measures. Provide 
institutional and unit-level data to support 
unit assessment activities. 

Individuals and units throughout the institution 
need ongoing professional development in 
assessment. Although the most intensive training 
needs to occur when the assessment system is 
introduced, it takes a few years for individuals to 
become proficient in assessment. In addition, the 
individuals responsible for overseeing assessment 
in the various units change over time, and it is 
necessary to train new individuals as well as provide 


refreshers to others. The IR office will need to 
organize systematic professional development, 
either by providing it or arranging for experts within 
the institution and/or external to the institution 
to provide it. Such assessment workshops may 
address the following topics: 

• introducing the institution's assessment 
system; 

• writing program outcomes; 

• determining viable direct and indirect 
measures; 

• developing rubrics for assessment; 

• using portfolios for assessment; and 

• closing the loop: making improvements 
based on assessment. 

It is very helpful to make available on the 
IR website assessment resources from the vast 
literature on assessment, including publications 
from the regional and specialized accrediting 
bodies on assessment and resources from other 
institutions. One such comprehensive resource 
from Schechter (n.d.) is posted on the North 
Carolina State University Planning and Analysis 
website at http://www2.acs.ncsu.edu/UPA/assmt/ 
resource.htm. 

IR should provide data that units need as part 
of their assessment activities or provide advice on 
collecting new data. For example, if an outcome 
deals with retention issues within an academic 
program, IR can provide the data that will help 
assess if the activities undertaken to improve 
retention have been successful. 

5. Summarize key findings from assessment 

that merit high-level consideration for action 

at the institution. 

The assessment information gathered across 
an entire institution is voluminous. While many 
assessment findings can be acted on by the 
units conducting the assessment, other findings 
may be at an institutional level and require the 
attention and action of the institutional leaders. 

IR or the assessment committee will need to 
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review assessment findings and ascertain the 
most important actionable issues that require 
institution-level attention and present them 
succinctly to the leadership, along with possible 
solutions. For example, if NSSE results indicate that 
students across disciplines feel they do not have 
adequate access to faculty members, this could 
be an issue that is discussed at an institutional 
level with faculty and academic administrators to 
determine what actions may be taken to address 
this concern. Because closing the loop by making 
improvements is essential to good assessment, IR 
needs to find effective means of bringing high-level 
findings to the attention of institutional leadership, 
making them part of the conversations surrounding 
resource allocations, and identifying strategic 
priorities. 

Accreditation Support 

Institutions of higher education in the 
United States engage in the peer-review 
process of accreditation for quality assurance 
and accountability. Accreditation takes three 
forms: national accreditation for special types of 
institutions, regional accreditation, and specialized 
or programmatic accreditation (WorldWideLearn, 
n.d.). All of these forms of accreditation are high- 
stakes activities, and an IR or IE office can play key 
roles in assisting the institution in these important 
endeavors. Regional accreditation places its 
stamp of approval on the entire institution and is 
granted by one of six regional accrediting bodies. 
Most institutions of higher education in the 
United States have regional accreditation. Some 
institutions that have a very focused curricula, such 
as theology, health education, or career education, 
are accredited by national accrediting associations. 
In addition, there are specialized accreditation 
agencies for discipline-specific programs and 
special types of accreditation that may be limited to 
those institutions offering such discipline-specific 
programs. These regional, national, and specialized 
accrediting bodies are, in turn, recognized by the 


U.S. Department of Education. Loss of institutional 
accreditation is a severe blow. It results in not only 
the loss of recognition by peers, but it is often fatal 
to an institution because it withdraws eligibility for 
Title IV funds (i.e., the students of the institution 
are no longer eligible for federal financial aid). 
Specialized accreditation is specific to a particular 
discipline, such as business, engineering, nursing, 
or education. The loss of specialized accreditation, 
even within a larger institution that maintains 
regional accreditation, can have a significant 
negative impact on the ability to recruit quality 
students and faculty and to attract external funding. 
In some cases, particularly in the health disciplines, 
it makes graduates of the program ineligible for 
licensure or certification. It is therefore imperative 
that the institution and academic programs 
seek and maintain both regional and specialized 
accreditation. Institutional effectiveness is an 
essential requirement in institutional accreditation 
and is discussed within the assessment section as 
well as within this section. 

Below are five activities in which institutional 
research can engage to help attain successful 
accreditation. 

1 . Conduct gap analysis for regional 

accreditation criteria and ensure a strategy to 

address gaps. 

In preparing for institutional accreditation, at 
least three years prior to the year the self-study is 
due, an institution should conduct an institution- 
wide gap analysis to determine if there are any 
areas of weakness or noncompliance in relation 
to the regional accrediting body's standards. This 
analysis can be conducted by an office, a select 
group of individuals, or a committee structure 
where subcommittees are responsible for clusters 
of standards. The organizational structure for 
institutional accreditation should be one that 
works for the particular institution, considering its 
strengths and organizational culture. Whatever 
the structure chosen, the effort, starting with 
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the gap analysis, must be well organized by 
an entity at the institution and must obtain 
comprehensive information from all facets of the 
institution, including academic, student support, 
educational support, and administrative units. 

For each standard, the unit organizing the effort 
must become familiar with what the accrediting 
body expects within each standard, and the 
documentation and information required. It is 
necessary for an individual or group of individuals 
leading the effort to attend one or more of the 
training sessions provided by the accrediting 
body to become familiar with the expectations for 
each standard. It is also very helpful to become 
familiar with the publications by the accrediting 
body regarding particular standards and with a 
few self-studies of other institutions that have 
successfully navigated recent accreditation visits 
by the same accrediting body. Armed with this 
information, a gap analysis should be conducted 
for each standard, identifying the following types of 
information: 

• Based on knowledge of the expectations of 
the standard, is the institution in compliance? 

• If so, what activities, policies, practices 
and documentation exist to demonstrate 
compliance? Where is this information 
available? 

• If not, what needs to be done in order to 
bring the institution into compliance? 

Develop an action plan for each area of 
weakness or non-compliance and obtain the 
support of the highest levels of leadership at 
the institution to implement the plan in a timely 
manner so that the institution is in full compliance 
prior to submission of the self-study. The most 
important standards, which can have a significant 
impact on the successful outcome of accreditation, 
merit special attention to ensure compliance. These 
core standards vary by the institutional accrediting 
body, but generally include governance, evidence 
of fiscal stability, institutional or educational 
effectiveness and assessment, and sufficiency of 


faculty. For example, the Western Association of 
Schools and Colleges (WASC), a regional accrediting 
body, defines two basic areas of core commitments: 
institutional capacity and educational effectiveness. 
An example of a format for a gap analysis is 
included in Appendix A. 

2. Establish a repository that is kept up to date 

and which provides, or indicates the location 

of, documents and evidence needed for 

regional accreditation. 

As one conducts a gap analysis, one becomes 
aware of the types of documents that will be 
needed to demonstrate compliance with each 
standard. Often the documents are scattered 
throughout the campus, may be incomplete or 
out-of-date, and reside with individuals who 
may not be at the institution by the time the self- 
study is developed. It now becomes necessary 
to begin developing an electronic repository of 
the documents that is readily accessible as the 
self-study is developed. The repository can be 
organized by standard and can also contain a library 
of documents to be referenced for more than one 
standard. Examples of general documents that may 
need to be referenced by several standards include: 

• The current mission statement of the 
institution; 

• The current strategic plan; 

• Minutes of meetings of the governing body; 

• Minutes of meetings of important 
committees or bodies such as the Faculty 
Senate; 

• The institution's policies and/or regulations; 

• The institution's procedures for various 
activities; and 

• Any other documents that have helped to 
shape the institution's current activities. 

Beyond these fundamental documents, each 
standard will also require additional specific 
documents. For example, the standards on 
assessment will require a database documenting 
assessment activities by each unit throughout the 
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institution. Generally, accrediting bodies require 
several cycles of assessment data; it will not be 
sufficient to provide data only for the year in which 
accreditation occurs. Such a database is discussed 
in greater depth in the section on assessment in this 
paper. 

It is important to not only establish the 
repository but also to keep it up to date so that 
by the time of the self-study and site visit, current 
documents continue to reside in the repository. 

If the repository is kept up even following the 
successful reaffirmation, it will be a very helpful 
tool for the institution's normal functioning and will 
make it much easier to mount the next reaffirmation 
effort. 

3. Make available necessary institutional and 

programmatic data for accreditation reports 

and program reviews. 

Self-studies for institutional accreditation usually 
require extensive data to demonstrate compliance 
with some standards. This may include both data 
about the institution itself as well as comparative 
data for peer institutions. It is important that IR 
provide these data so that they reflect official 
institutional data and are consistent with data 
the institution has reported to IPEDS and other 
national reporting systems. Accreditation teams 
may review and compare data in the self-study with 
data reported by the institution elsewhere, and 
inconsistencies will raise questions. IR can provide 
data that bolster arguments made in the self-study 
that the institution is in compliance with standards 
such as listed below. 

• Data that demonstrate the institution is 
meeting the various components of its 
mission; 

• Data that demonstrate that the institution is 
educating and graduating students at rates 
comparable to or better than its peers; 

• Data on enrollment and graduation; 

• Data on sufficiency of faculty and their 
qualifications; 


• Data on financial status of the institution; and 

• General information for the institution, such 
as historic information, governance structure, 
institutional policies and procedures such as 
those governing admission, graduation, and 
transfer of students. 

The data required from IR for specialized 
accreditation may be less extensive than for 
institutional accreditation. The data may include 
some institution-level data as well as data specific to 
the program seeking accreditation and peer data for 
the particular program. Again, it is important that 
the data be provided from IR so that they reflect 
official institution data that are consistent with 
data reported to external entities such as IPEDS, 
rather than relying on data generated internally by 
the program alone. Programs may use definitions 
that vary from data definitions in common usage 
and end up with data that are inconsistent with 
nationally reported data. If the accrediting body 
specifies particular data and definitions of such 
data, IR could work collaboratively with the program 
to generate the data required. 

Another means of assisting academic 
departments to meet institutional accreditation 
requirements is by helping them conduct program 
reviews. Program reviews became widely used as 
quality assurance activities in the United States 
beginning in the 1 970s (Bogue & Saunders, 1 992). 
Since that time they have evolved as an important 
component in demonstrating institutional 
effectiveness to accrediting bodies. The espoused 
purposes of program reviews in many public 
and private institutions now reflect increased 
institutional aspirations, accountability, and a focus 
on student learning. While faculty committees 
may be conducting the academic program reviews, 
many IR or IE offices provide the appropriate data 
and may coordinate the program review activities of 
their institutions. Program reviews may take many 
forms, ranging from a quantitative emphasis to a 
qualitative emphasis in judging the viability, quality, 
and accountability of programs. The reviews may 
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also involve external consultants who are experts 
in the disciplines being reviewed or internal 
institutional committees that review the program's 
performance on a set of standards much like 
specialized accrediting bodies. In fact, for programs 
that undergo specialized accreditation, it may 
be advisable for such accreditations to serve the 
purpose of program review, rather than engaging in 
a duplicative activity. A discussion of various forms 
of reviews and how one may select a methodology 
best suited for one's own institution may be found 
in the AIR Professional File article, "Program Review: 

A Tool for Continuous Improvement of Academic 
Programs" (Pitter, 2007). 

4. Collect and update annually, information on 
regional and specialized accreditation and 
upcoming accreditation site visits. 

While regional and national accreditation is 
addressed at the institutional level, specialized 
accreditation is usually best addressed by the 
academic unit undergoing that accreditation. 
Therefore, a dean or department head would be 
responsible for specialized accreditation activities. 
However, it is important that the institutional 
leadership is aware of all the major accreditation 
events at the institution. To this end, a central 
office, such as IR, should maintain on an annual 
basis, a listing of all accreditations at the institution, 
the period of accreditation, when the next self- 
study and site visit would be due, and plans for 
seeking any new accreditation. A sample format for 
such a document, utilized by the State University 
System of Florida, is included in Appendix B. A 
document such as this ensures that the university 
leadership is kept aware of upcoming specialized 
accreditations each year and is not caught by 
surprise. In the absence of such information, the 
unexpected departure of a key individual in an area 
scheduled to undergo reaccreditation, or lack of 
information on the part of key individuals in the 
area, could lead to disastrous results. It also enables 
the institutional leadership, such as the Provost, 


to engage in proactive discussions with the units 
scheduled for accreditation visits to anticipate 
and address any issues. The units undergoing 
specialized accreditation should conduct their 
own gap analyses, similar to that conducted by the 
institution for regional accreditation, and should 
seek the assistance of institutional leadership or 
other offices when needed. 

5. Establish appropriate university processes 

for accreditation. 

The development, review, and submission 
of the self-study for regional accreditation is 
a complex, comprehensive, time-consuming 
activity that must be well-organized at the 
institutional level following the completion of the 
gap analysis described earlier in this paper. The 
development of a traditional 1 0-year review self- 
study typically spans approximately 12-18 months. 
The responsibility for the writing of the self study 
should be deliberately assigned, by standard, to 
individuals or units most knowledgeable about that 
standard. Clear timelines should be established for 
writing, review by multiple individuals, revisions, 
approval by the institutional leadership (usually 
signed by the President), and timely submission. It 
should be noted that some accrediting bodies are 
moving to seven-year reporting cycles that require 
more continuous reporting within a shorter cycle. 
Additionally, the Academic Quality Improvement 
Program (AQIP) provides a systematic process 
to accreditation that reflects the principles of 
continuous improvement (AQIP, n.d.). Self-studies 
for institutional accreditation include extensive 
documentation and, therefore, are now generally 
submitted electronically. An electronic format 
should therefore be developed for the self-study 
and documentation, following guidelines by the 
accrediting body, must be followed by all the 
authors of the self-study. Integrity is an important 
aspect of the self-study, and some accrediting 
bodies, such as SACS, even have a standard relating 
to institutional integrity. It must be made clear at 
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the outset, to all parties concerned, that veracity 
must be maintained throughout the self-study. If a 
gap analysis has been conducted well in advance, 
and weaknesses addressed, then an institution 
can, with integrity, claim to be in compliance with 
all standards and demonstrate that compliance 
in the self-study. The document should undergo 
thorough review by individuals who are familiar 
with the expectations of the regional accreditor. 

The process for specialized accreditation is 
more decentralized. The self-study is developed by 
individuals in the unit undergoing accreditation. 

At large institutions, the entire operation of 
preparation for reaccreditation by a specialized 
accrediting body may be conducted at the unit 
level with little oversight at the institutional level. 
But at the outset, the institution should have 
clear procedures and expectations related to the 
development of the self-study, its review, and, if 
required, approval by the President or designee. The 
procedures could cover the following: 

• Institutional process for seeking initial 
accreditation (i.e., this decision should be 
made in consultation with the Provost 
and/or President, not by the unit seeking 
accreditation alone); 

• Expectations for performance of a gap 
analysis and the development of the self- 
study, including integrity and not being 
overly negative in the self-study merely in 
order to gain unrealistic resources from the 
institution; 

• Any required institution level review of self- 
study prior to submission; 

• The process for scheduling meetings 
between the Provost and/or President during 
the course of the site visit, if required; 

• Process for responding to the site visit team's 
report and final report and action from the 
accrediting body; and 

• Procedures for making accreditation findings 
and final actions public, whether they be 
positive or negative. 


If the institution's history of specialized 
accreditation is not strong (i.e., programs have a 
history of being found noncompliant with several 
standards), it may be helpful to institute a discussion 
of the findings of the gap analysis between a central 
office, or even the Provost, and the head of the unit 
prior to development of the self-study. It may also 
be prudent to have an institution-level review of the 
self-study prior to submission. 

Conclusion 

Demand for accurate, timely, accessible, and 
understandable data, along with value-added 
information and analysis remains strong in today's 
higher education environment of accreditation, 
assessment, and reporting. Although the need and 
demand for data are clear, how to best structure an 
institution's data offices remains less clear. There is 
often overlap and similarity between the functions 
of various offices charged with data collection, 
storage, analysis, and reporting, in particular, among 
the offices of Institutional Research, Institutional 
Effectiveness, and Information Management. 
Conversely, there is a stream of thought that the 
functions between these three offices are distinct 
and should therefore be located in separate offices. 

This paper argues that there is indeed a 
connection between the responsibilities of IR, IE, 
and IM and that colleges and universities would 
be well-suited to recognize these connections 
and take steps to ensure synergy among data and 
information activities. The paper has outlined the 
main functions and activities that, if integrated, 
would create an effective infrastructure for all data 
and information needs of a college or university. 

The activities outlined are the essential IR, IE, and 
IM undertakings that would aid an institution to 
function effectively and meet its mission. 

Addressing these activities synergistically, either 
by combining them under one umbrella office or by 
creating formal and informal links that ensure close 
collaboration creates an environment that positions 
an institution for success by assuring that important 
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information and performance measures do not fall 
through the cracks of isolated offices. For example, 
IM data that reveal a single important issue such 
as declining graduation rates could generate 
activities in all five foundations and across all three 
areas. IR could undertake an analytical study to 
suggest possible causes of the decline; assessment 
activities in the general education core and in 
academic programs could examine possible issues 
contributing to the declining graduation rate and 
suggest actions to reverse the trend; the issues may 
rise to the level of needing to be addressed in the 
strategic plan, and peer data could place the issue 
in context; and, addressing the issue may assist 
in meeting accreditation standards. Examining 
an issue through multiple lenses bringing their 
focused power to bear on the issue is more likely 
to reveal effective ways of addressing it rather 
than fragmented and isolated approaches. The 
synergy between the activities also aids university 
leadership by framing issues and solutions 
holistically so that the interconnections and 
implications are made explicit for them, reflecting 
more realistically the complex relationships 
inherent in academic issues. 


Note: The following list of references and bibliographic 
resources was compiled to support this study. While 
many are not cited directly in this paper, they are 
included because the authors felt that they provide 
important information to facilitate a deeper 
understanding of the topic and support future 
research on the issues in this study. 
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